Method and system for improved per-pixel shading in a computer graphics system

ABSTRACT

A method for shading polygon surfaces in a real time rendering system. Providing at least one polygon surface to be shaded. The polygon surface having a plurality of pixels and including at least one surface angle. Providing at least one point light source. Calculating using computer hardware, for substantially each drawn pixel of said polygon surface, a substantially normalized 3D surface direction vector and a 3D point light vector.

BACKGROUND OF THE INVENTION

[0001] The present invention relates to the field of per-pixel lightingin real-time three-dimensional (“3D”) computer graphics hardware andsoftware. Primarily, most real-time computer graphics systems rely onper-vertex lighting schemes such as Gouraud shading. In this scheme, thecurvature of a polygon surface is represented through different surfacenormal vectors at each polygon vertex. Lighting calculations are carriedout for each vertex and the resultant color information is interpolatedacross the surface of the polygon. Lighting schemes such as Gouraudshading are generally utilized for their speed and simplicity ofoperation since they require far less calculation than more complexstrategies. Per-pixel lighting, in contrast, is a lighting strategy inwhich separate lighting calculations for one or more light sources arecarried out for each pixel of a drawn polygon. Most well-known per-pixellighting strategies are variations on a basic vertex normalinterpolation scheme, i.e., Phong shading. Vertex normal interpolationstrategies interpolate the normal vectors given at each vertexthroughout the polygon surface. For each pixel, the interpolated vertexnormal is normalized to unit length and then used in per-pixel lightingcalculations. Typically the per-pixel calculations involve taking thedot product of the normal vector and the light source vector to arriveat a light source brightness coefficient. While fast per-pixel dotproduct hardware is not infeasible with the speed and complexity oftoday's microprocessors, the calculations involved in normalizing theinterpolated vertex vector (i.e., floating point square root anddivision) are prohibitive for practical real-time implementation at highspeed.

[0002] Another per-pixel lighting technique, commonly referred to asbump mapping, involves using a two-dimensional (“2D”) map to storesurface height or orientation and using texel values from this map toperturb a (usually interpolated) surface normal vector. Calculation intraditional combinational bump mapping (i.e., where the bump map angleperturbation is combined with a potentially changing surface normal)mostly involves resolving the bump map perturbation to a 3D vector thatis subsequently combined with the surface normal vector. Since thesurface normal vector may change from pixel to pixel, an appropriate,usually orthogonal, orientation must be given to the bump map vector.This process usually requires additional normalization and a significantcomputational overhead, making combinational bump mapping approachesimpractical for efficient real-time calculation. A well-known method ofavoiding these calculations is to store a bump map as a collection ofnormalized 3D vectors, therefore avoiding the need for normalization andcombination. While this strategy is more practical for real-timeimplementations, it has several drawbacks. Such a system is inflexiblesince bump maps may only be used for objects in preset orientations, andsurface curvature must be represented within in the bump map rather thanthrough vertex normals as in Phong shading and its equivalents.Furthermore, the accuracy of the image is limited by the granularity ofthe bump map, since values failing between adjacent texels aretraditionally interpolated but not re-normalized. Another drawback ofthe above-mentioned bump mapping scheme is the size and inflexibility ofthe bump maps. Since the bump map texels contain 3D vectors, medium tolarge complexity maps will occupy a great deal of memory. Also, due tothe specific nature of the bump maps, they are generally only usable onthe surfaces for which they were designed; therefore such bump maps arenot often used for multiple surfaces.

[0003] A further aspect of per-pixel lighting is the calculation ofintensity of specular reflections. Traditionally, the calculation ofspecular reflection involves the dot product of the light source vectorand the view reflection vector (the view, or eye, vector reflectedaround the surface normal vector). Alternately, the same calculation canbe made with the dot product of the view vector and the reflection ofthe light vector around the normal. In either of the alternatives, atleast one vector must be reflected around a surface normal vector thatpotentially changes from pixel to pixel. The calculation required toobtain a reflected vector, while not as costly as bump map combination,is nonetheless significant.

[0004] Yet another complication in per-pixel lighting is presented bythe cases of point light sources and point view vectors. Point lightsources involve a light vector that changes on a per-pixel basis.Traditionally, the difference vector between the surface point and thelight source is calculated and normalized for each pixel, which iscomputationally undesirable for efficient calculation. Likewise, pointview vectors involve a view vector that changes on a per-pixel basis.Utilizing point view vectors also requires the calculation andnormalization of a difference vector on a per-pixel basis.

[0005] The application of the aforementioned per-pixel lightingtechniques provides visually enhanced, higher quality and more realisticimages than today's real-time image generators are capable of producing.While techniques exist which can provide similar images, thesetechniques are difficult to implement and inflexible to use. Therefore,there exists a real need for a practical and efficient apparatus andmethod that provides vertex normal interpolation, combinational bumpmapping, specular reflection calculation, and support for point lightingand point viewer within real-time 3D graphics systems.

SUMMARY OF THE INVENTION

[0006] The present invention is directed to a method for shading polygonsurfaces in a real time rendering system. The method includes the stepof providing at least one polygon surface to be shaded. The polygonsurface having a plurality of pixels and including at least one surfaceangle. The method also includes the step of providing at least one pointlight source. The method further includes the step of calculating usingcomputer hardware, for substantially each drawn pixel of said polygonsurface, a substantially normalized 3D surface direction vector and a 3Dpoint light vector.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

[0007]FIG. 1 is a diagram illustrating the translation of normal vectorsto a view coordinate system in accordance with a preferred embodiment ofthe invention.

[0008]FIG. 2 is a diagram illustrating the conversion of a 3D vectorinto an angle-proportional 2D vector in accordance with a preferredembodiment of the invention.

[0009]FIG. 3 is diagram illustrating the combination of a surface anglevector and a bump map vector to produce a composite surface angle vectorin accordance with a preferred embodiment of the invention.

[0010]FIG. 4 is a diagram illustrating the production of a viewreflection vector from a composite surface angle vector in accordancewith a preferred embodiment of the invention.

[0011]FIG. 5 is a diagram illustrating the calculation of the viewreflection vector.

[0012]FIG. 6 is a diagram of a preferred hardware embodiment of thepresent invention.

[0013]FIG. 7 is a diagram illustrating an AP translation unit inaccordance with a preferred embodiment of the invention.

[0014]FIG. 8 is a diagram illustrating a preferred hardware embodimentof the per pixel operation of the present invention.

[0015]FIG. 9 is a diagram illustrating the preferred embodiment of thepoint light operations of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

[0016] The present invention provides a method and system for theefficient calculation of complex per-pixel lighting effects in areal-time computer graphics system. For the purposes of this disclosure,the term “real-time computer graphics system” is defined as any computerbased system capable of or intended to generate images at a rate greaterthan or equal to 10 images per second. Some examples of real-timecomputer graphics systems include: stand-alone console videogamehardware, 3D graphics accelerator cards for PC's and workstation classcomputers, multipurpose set top boxes, virtual reality imaging devices,and imaging devices for commercial or military flight simulators. All ofthe above-mentioned systems are likely to benefit from the increasedimage quality afforded by the methods and practices of the presentinvention.

[0017] As used herein, the term “angle-proportional” is defined as acharacteristic of a 2D vector wherein the length of the 2D vector isproportional to the angle between a 3D direction vector (correspondingto said 2D vector) and a 3D axis vector (usually representing the z-axisof a pre-defined coordinate system).

[0018] As also used herein, the term “view coordinate system” is definedas a 3D coordinate system (which can be defined by a 3D position vectorand at least three 3D direction vectors) that represents the positionand orientation from which a 3D scene is being viewed.

[0019] As further user herein, the term “view vector” is defined hereinas a 3D vector representing the forward direction from which a scene isbeing viewed. The view vector is usually directed, either positively ornegatively, along the z-axis of the view coordinate system and isexpressed in world-view coordinates.

[0020] As further used herein, the term “current polygon” is definedherein as the polygon that is currently being operated on by the methodsof the present invention.

[0021] Lastly, as used herein, the term “current pixel” is definedherein as the pixel within a polygon surface currently being operated onby methods of the present invention.

[0022] The present invention comprises two areas of execution within acomputer graphics system: per-polygon operations and per-pixeloperations. The per-polygon operations of the present invention areperformed once for each polygon in a scene to which the presentinvention is applied. Likewise, the per-pixel operations of the presentinvention are performed for each drawn pixel on a polygon surfacewherein the aforementioned per-polygon operations are assumed to havebeen previously applied to said polygon. Additionally, the presentinvention provides a method to enable accurate real-time calculation ofpoint light vectors useful for advanced lighting strategies. Most of theper-polygon and per-pixel operations of the present invention aredetailed in U.S. patent application Ser. No. 09/222,036 filed on Dec.29, 1998, in the name of David J. Collodi, the disclosure of which ishereby incorporated by reference. The operations are detailed herein forpurposes of consistency and example.

[0023] The per-polygon operations of the present invention are performedin order to provide a set of angle-proportional surface angle vectors tobe utilized within the per-pixel operations. For the purposes ofsimplicity, this disclosure shall assume the existence of a polygon tobe rendered wherein said polygon provides a 3D surface normal vector foreach of its vertices and said polygon is the current polygon. Thesurface normal vectors are used to collectively specify the amount ofcurvature along the polygon surface.

[0024] First, the surface normal vectors of the current polygon arerotated to correspond to the direction of the view coordinate system. Itis well known in the art that a 3D coordinate system (or rather thetranslation to a particular 3D coordinate system) can be represented bya 4×4 matrix. A 4×4 matrix can represent both rotational and positionaltranslations. Since the rotation of surface normal vectors requires onlyrotational translations, a 3×3 matrix, M, is used. Each surface normalvector, N_(i), is multiplied by matrix M to produce a correspondingrotated surface vector, R_(i).

M*N _(i=R) _(i)  (1)

[0025] The above calculation is performed for each surface normal vectorbelonging to the current polygon, N₁-N_(x) where x is the number ofvertices in the polygon. After this calculation is performed for eachsurface normal vector (i.e., for each vertex), a set of rotated surfacevectors, R₁-R_(x), is produced. The purpose of performing this rotationto view coordinate space is to provide a common orientation to allpolygons in the rendered frame. Additionally, all 3D directional andpoint light source vectors to be used in the lighting of the currentpolygon must be translated to the view coordinate system as well. Thetranslation of said 3D light source vectors is accomplished by the same,above-mentioned, translations used to translate the surface normalvectors. The result of the 3D light source vector translation is a setof corresponding 3D rotated light vectors that are expressed relative tothe view coordinate system.

[0026] Next each rotated vector, R_(i), is transformed to anangle-proportional 2D surface angle vector, n_(i), where the length ofn_(i) is proportional to the angle between R_(i) and its z-axis. Aprocedure for the transformation of a 3D vector into a correspondingangle-proportional 2D vector is detailed in the above-identified CollodiU.S. patent application Ser. No. 09/222,036. After transforming each Rvector, a set of 2D surface angle vectors, n₁-n_(x), are created. Each nvector is angle-proportional to its corresponding R vector. FIG. 1demonstrates a translation of normal vectors, 6, to a view coordinatesystem. The resulting R vectors 10 and corresponding n vectors 12 areillustrated. As a result of the transformations detailed in Collodi U.S.patent application Ser. No. 09/222,036, an angle-proportional vectorlength of 1.0 corresponds to an angle of 90° between the original 3Dvector and its z-axis. This is demonstrated by FIG. 2, wherein thevector 22 is converted to the angle-proportional 2D vector 24 where thelength of the 2D vector, 26, corresponds to the angle between theoriginal vector and its axis vector 18. The preceding scale ofangle-proportional vectors will be used herein for purposes of clarityand example only. Those of ordinary skill in the art should recognizethat the resultant angle-proportional vectors can be transformed intoany arbitrary 2D coordinate system. Furthermore, although the presentdisclosure presents angle-proportional vector values in floating pointformat, this is done for purposes of example only and it may be moreefficient in practice to work with angle-proportional vectors in afixed-point format. For example, a value of 1.0 could be represented as256 in a fixed-point format, whereas 2.0 is represented as 512 and soon. It is a preferred practice of the present invention to deal withfixed-point 2D vectors.

[0027] An optional step is to limit the vectors that are far from theview angle. The direction of vectors at or near 180° from the viewer isunstable. It is therefore advantageous to limit the direction anddistance of these vectors. An example of a basic limiting method isdetailed in the following disclosure. First, a 3D vector U is obtainedwhere the direction of U is perpendicular and normal to the plane of thepolygon (i.e., U is the “real” polygon surface normal vector). Next thex and y components of U are scaled by dividing each component by thelarger component (either x or y) of U. Then the scaled x and ycomponents of U are doubled. The scaled x and y components of U form 2Dvector u which represents the angle-proportional direction of thepolygon surface at (or slightly greater than) 180°. Angle-proportional nvectors with large angles relative to the viewer (which can easily bederived from the z-coordinate of the corresponding R vector) areinterpolated with the u vector weighted on the relative angle (toviewer) of the n vector.

[0028] A further optional step at this point is to calculate a2-dimensional bump map rotation value. Since a bump map, in whateverformat it is presented, is basically a 2D texture map, the map itselfhas its own local coordinate system, i.e., which direction is up, down,left, right, etc. The bump map is mapped arbitrarily onto the polygonsurface and therefore may not necessarily share the same orientation asthe view coordinate system. Since bump map perturbations will be done in2D space, only a 2D rotation value is necessary to specify the 2Drotation of the bump map coordinate system relative to the viewcoordinate system. A simple method of obtaining said bump map rotationis to perform a comparison of the bump map orientation (using the bumpmap coordinate values provided at each polygon vertex) to the screenorientation of the translated polygon (since the screen orientationcorresponds directly to the view coordinate system). Two 2D bump maprotation vectors are required to specify the translation from the bumpmap orientation to the view orientation. The use of any known techniquesto obtain said 2D vectors is acceptable. In one embodiment of thepresent invention, the bump map orientation vectors are used to rotateeach of the above mentioned 2D surface angle vectors, n₁-n_(x), to thebump map orientation. Additionally the aforementioned 3D rotated lightvectors must also be rotated (in the x-y plane) to the bump maporientation. This is accomplished by applying the bump map rotationaltranslations to the x and y coordinate of each 3D rotated light vector.An alternate embodiment uses (the inverse) of the bump map orientationvectors to translate 2D bump map vectors into the view coordinate systemas opposed to translating surface angle and light vectors to the bumpmap coordinate system.

[0029] The next section details the per-pixel operations of the presentinvention. As previously stated, the per-pixel operations of the presentinvention are performed for at least each visible pixel on the screensurface of the current polygon during the scan-line conversion of thepolygon. Note that the per-pixel operations detailed herein need not beperformed concurrently with the drawing of the current polygon to videoRAM. Alternate embodiments of the present invention perform per-pixellighting operations prior to final rendering to screen memory.Additional embodiments of the present invention perform per-pixellighting operations after color values have been placed in screenmemory. It is, however, a preferred method to perform per-pixel lightingoperations concurrently with the drawing of color values to screenmemory.

[0030] Initially, the previously mentioned set of 2D surface anglevectors is interpolated from their vertex values, n₁-n_(x) as previouslydefined, to the location of the current pixel. Techniques forinterpolating vertex values to an arbitrary point within a polygon arewell known to those skilled in the art. Any interpolation strategy canbe used including, but not limited to, linear interpolation, inverse(perspective-correct) interpolation, quadratic or cubic interpolation.The interpolation of 2D surface angle vectors produces an aggregate 2Dsurface angle vector, n, which represents the orientation of the polygonsurface at the current pixel. In circumstances where the orientation ofthe polygon surface does not change, i.e., flat surfaces, it is notnecessary to interpolate the n value from given vertex values since allvertex values would be the same. In this case, a fixed n value may beused which is generally, although not necessarily, equivalent to thenormal surface orientation of the current polygon.

[0031] Next, the aggregate surface angle vector, n, is combined with a2D bump map vector, b. In one embodiment of the present invention, thebump map vector is obtained from a given bump map and accessed byinterpolated bump map coordinates given at the polygon vertices inaccordance with standard vertex mapping techniques well-known by thoseskilled in the applicable art. The 2D bump map vector may be obtaineddirectly from the texel values stored in the bump map. Alternately, the2D bump map vector may be calculated from retrieved texel values storedin the bump map. One well-known example of said bump map vectorcalculations is storing relative height values in the bump map. Heightvalues are retrieved for the nearest three texel values. Assuming thatthe texel at coordinates x, y (t(x,y)) maps to the current pixel, thentexels t(x,y), t(x+1,y), and t(x, y+1) are loaded from the bump map.Since each texel contains a scalar height value, the 2D bump map vector,b, is calculated from the differences in height values in the followingmanner:

b=(t(x+1,y)−t(x,y),t(x,y+1)−t(x,y))  (2)

[0032] An alternate method for storing bump map data involves storing apolar representation of the bump map vector at each texel. The polarrepresentation comprises two fields, one for the 2D angle of the bumpmap vector and another for the magnitude of the bump map vector. Apreferred method of retrieving the 2D bump map vector from said polarrepresentation is through the use of a lookup table. The direction andmagnitude (or functions of those values) values are used to index alookup table which returns the appropriate 2D bump map vector. Theprimary advantage of storing bump map vectors in polar representation isthat the rotation of polar vectors is easily accomplished. In theaforementioned embodiments in which the bump map vector is rotated toview orientation, said rotation is facilitated by storing bump mapvectors in polar representation. Rotating a polar vector involvesproviding a scalar angle of rotation (for example an 8-bit number wherethe value 256 is equivalent to 360°) and simply adding that number tothe rotation value of the polar vector.

[0033] For added image quality, map based bump map values may beadditionally interpolated with any well-known texel interpolation schemesuch as bi-linear or tri-linear interpolation. For direct mappingschemes, i.e., where texels contain 2D bump map vectors, the vectorvalues given at each texel are interpolated. Alternately for indirectmapping schemes, such as the height map detailed above, it is desirableto first calculate all necessary 2D bump map vectors and subsequentlyinterpolate those vectors. It should be noted that one or more 2D bumpmap vectors may be combined to produce the final b vector. The abilityto easily combine and aggregate multiple bump maps and/or to combinebump map perturbation with a variable surface normal is an advantageousfeature of the present invention since this technique provides for agreat deal of flexibility, reusability and decreased memory costs inmany 3D graphics applications.

[0034] In an alternate embodiment of the present invention, 2D bump mapvalues are calculated procedurally from a function of the surfaceposition (and other optional values). Procedural texture/bump maptechniques offer the advantages of flexibility and minimal memory usagebalanced with the cost of additional calculation. Alternately, if bumpmapping is not selected for the current polygon, a null b vector (0,0)can be used. In this case, it is not necessary to combine the bump mapvector with the n vector and the combination step may therefore beskipped. For the purposes of clarity and continuity of the exampledetailed herein, a b vector of (0,0) will be used for cases in whichbump mapping is not used.

[0035] Once the bump map vector, b, is arrived at, it is combined withthe n vector through vector addition to produce the composite surfaceangle vector, c:

c=n+b  (3)

[0036] The c vector represents the composite orientation of the polygonsurface at the current pixel with respect to polygon curvature and bumpmap perturbation. FIG. 3 demonstrates the combination of surface anglevector n 28 and bump map vector b 30 to produce the composite surfaceangle vector c 32. In alternate embodiments, the aforementioned c vectoris used to address an environment map. Environment maps aretraditionally 2D color maps that provide reflection information for agiven scene. Since the c vector represents the composite orientation(due to surface bump and curvature) of the current pixel in relation tothe view coordinate system, it can be used to accurately address a 2Denvironment map that is also (traditionally) relative to the viewcoordinate system. By addressing an environment map in this manner, aconsistency is maintained between lighting equations and reflectionvalues (from an environment map). A significant feature of the presentinvention is the provision of a method for coordinating traditional,equation-based, lighting information with reflection (environment map)values in a real-time 3D graphics system.

[0037] Once the c vector is arrived at, the view reflection vector isnext calculated. The view reflection vector represents the direction theview vector reflects off of the surface at the current pixel. Since the2D vector coordinate space is angle-proportional to the view vector, thedirection of the view vector is located at coordinates (0,0).Consequently, the 2D view reflection vector, r, reflected around the cvector (which represents the current pixel surface orientation) issimply the c vector doubled:

r=2c  (4)

[0038]FIG. 4 illustrates the production of view reflection vector r 34from composite surface angle vector c 36.

[0039] The above calculation is accurate provided that the direction ofview is always directed along the z-axis of the view coordinate system.For most applications, this assumption is accurate enough to producevisually sufficient results. However, the exact view direction varies inaccordance with the screen position of the current pixel since itsscreen position represents an intersection between the view plane andthe vector from the focal point to the object surface. The precedingscenario in which the view direction is allowed to vary with screencoordinates is commonly referred to as a point viewer. In cases in whichpoint viewing is desired, the view reflection vector, r, must becalculated in an alternate manner. First the 2D displacement vector ofthe screen coordinates of the current pixel and the screen coordinatesof the center of the screen must be found. Assuming the screencoordinates of the current pixel are represented by 2D vector p, and thescreen coordinates of the center of the screen are represented by 2Dvector h, the 2D displacement vector, d, is calculated as follows:

d=p−h  (5)

[0040] Next, 2D displacement vector d is converted to an approximatelyangle-proportional 2D offset vector, o. The most straightforward way toconvert d to o is to multiply d by a scalar value, y, representing theratio of the viewing angle to the screen width. The viewing anglerepresents the total angle from the focal point to the two horizontal(or vertical) edges of the screen and should be given in the sameangle-proportional scale as other angle-proportional vectors (in thisexample, a value of 1.0 representing 900). The screen width is just thewidth (or height) of the screen in pixels. For example, if the viewingangle is 45° and the screen is 100 pixels wide, the y value would be0.5/100, or {fraction (1/200)}. The o vector is calculated as follows:

o=d*x  (6)

[0041] In order to calculate view reflection vector, r, in the case of apoint viewer, the r vector is positively displaced by o. The formula forr is:

r=2c+o  (7)

[0042] The calculation is illustrated in FIG. 5, where view vector r 38is found by doubling vector c 42 and adding vector o 40.

[0043] It should be noted that the above formula is only anapproximation of the true view reflection vector. However, theapproximate view reflection calculated by the preceding formula is ableto produce visually consistent and convincing images with little or nodiscernable loss in image quality. In alternate embodiments of thepresent invention, the r vector, as opposed to the c vector, is used toaddress an environment map as previously detailed.

[0044] Once the 2D composite surface angle vector and view reflectionvector are calculated, they are next transformed into normalized (unitlength) 3D vectors. The 2D composite surface angle vector, c, istransformed into normalized 3D composite surface vector C. Likewise, 2Dview reflection vector, r, is transformed into normalized viewreflection vector A. The conversion from a 2D angle-proportional vectorto a normalized 3D vector by mathematical calculation is computationallyexpensive in terms of hardware complexity and computation time.Therefore, it is a preferred practice of the present invention toperform said conversion from 2D angle-proportional vector to normalized3D vector with the aid of a lookup table. The use of a lookup tableoffers the advantage of being able to produce normalized compositesurface and reflection vectors without using a square root operation.The complexity of the square root operation combined with the difficultyof calculating 3D composite surface and view reflection vectors hasheretofore prohibited practical real-time calculation of complexlighting effects. Methods of the present invention using lookup tables,therefore, represent a significant improvement in the real-timecalculation of complex per-pixel lighting effects.

[0045] A preferred lookup table method is to use fixed point x and ycoordinates of an angle-proportional vector to directly access a 2Dlookup table wherein said lookup table contains normalized 3D vectors.The vectors contained in the lookup table may be stored in eitherfloating point or fixed-point format. For matters of efficiency,however, it is a preferred practice of the present invention to store 3Dlookup table vectors in fixed-point format. For example, a fixed-pointformat of 8 bits per vector component, i.e., 24-bits per 3D vector,would provide sufficient accuracy while minimizing the size of thelookup table. Fixed point 3D vectors obtained from the lookup table caneasily be converted to floating point format for further calculation ifnecessary. In order to further enhance visual consistency, lookup tablevectors can be interpolated using any of a number of well-knowninterpolation techniques including, but not limited to, bi-linear andtri-linear interpolation, quadratic interpolation and cubicinterpolation. The size of the lookup table can be additionallydecreased due to the fact that the coordinate system is symmetric aboutthe x and y axis. Therefore the lookup table need only cover thepositive x/positive y quadrant. To utilize such a lookup table, negativex and y coordinates (in the 2D vector used to address the table) arefirst negated and the 3D vector is retrieved (and optionallyinterpolated) from the table. Then the corresponding x and/or ycoordinates in the 3D vector are negated provided that the x and/or ycoordinates of the 2D addressing vector were originally negative. Sinceseveral vector additions may be performed on angle-proportional vectors,the final c and r vectors can have lengths greater than 2.0 (equivalentto 180°). Therefore, the 2D lookup table must at least cover coordinatevalues ranging from 0 to 2.0. A 512×512 map should be of sufficientaccuracy to cover such a range, however larger maps may be implementeddepending on the desired accuracy.

[0046] An alternate embodiment of the present invention utilizes aone-dimensional lookup table. The lookup table is addressed by thesquare of the length of the above-mentioned addressing 2Dangle-proportional vector. Each lookup table element contains twoelements: a z-value and a scalar value s. The z-value is used as thez-coordinate for the resultant 3D vector while the s value is used toscale the x and y values of said addressing 2D vector yielding the x andy values of said resultant 3D vector. The above-mentionedone-dimensional lookup table strategy provides a significant memorysavings over the aforementioned 2D lookup table, but also incurs ahigher computational cost.

[0047] The lookup table strategies detailed above are presented for thepurpose of example only and, as can be recognized by someone skilled inthe applicable art, any adequate lookup table strategy may be employedwithout departing from the scope of the present invention as defined bythe appended claims and their equivalents.

[0048] Regardless of the calculation method applied, the conversion of2D vectors c and r to normalized 3D vectors produces unit-length 3Dcomposite surface vector C and unit-length 3D view reflection vector A.The C and A vectors can then be used in calculating diffuse and specularlight coefficients for any number of light sources. Given a light sourcewhose direction is represented by unit-length light source vector L, thediffuse coefficient, Cd, of said light source at the current pixel isgiven by:

c _(d) =L*C  (8)

[0049] While the specular coefficient, c_(s), is given by:

c _(s) =L*A  (9)

[0050] The specular coefficient value c_(s) is optionally applied to aspecularity function to account for surface reflectivitycharacteristics. For example, a commonly used specularity functionraises the c_(s) value to a given power, exp, where the higher expvalues produce “shinier looking” specular highlights.

[0051] A further alternate embodiment utilizes a one-dimensional lookuptable as in the previously mentioned lookup table strategy. As with theaforementioned strategy, a z-value and scalar s value are provided bythe lookup table. In this embodiment, however, the s value is not usedto scale the x and y values of the addressing vector. Rather, theaddressing vector, with the aforementioned z-value included, is used asa 3D vector in the above mentioned diffuse and/or specularity dotproduct calculation. The result of the dot product calculation is thenscaled by the s vector to produce the correct shading value as in thefollowing equations:

c _(d)=(L*C)*s  (10)

c _(s)=(L*A)*s  (11)

[0052] Once diffuse and specular components have been calculated, theymay be used as scalar values to apply diffuse and specular lighting tothe current pixel. Standard color based pixel lighting algorithmsutilizing scalar light coefficients are well-known to those skilled inthe art. Any such lighting algorithm (which requires scalar diffuse andspecular coefficient values) may be applied to modulate the color of thecurrent pixel.

[0053] A further aspect of the present invention applies to thecalculation of point light source direction vectors. As opposed todirectional light sources, where the light source direction is constantwithin the frame, the direction of point light sources is variableacross a surface. The direction at which a point light strikes a surfaceis determined by difference between the position of the surface and thelight source. A prior art approach to the calculation of point lightsource direction vectors involves normalizing the difference vectorbetween the light source position and the surface position. Sincestandard vector normalization requires computationally expensivedivision and square root operations, the application of said approach tothe calculation of point light source direction vectors is infeasiblefor efficient real-time operation. A method is presented for theaccurate calculation of point light source direction vectors that doesnot involve division or square root operations.

[0054] According to the present invention, a 3D difference vector, D, isobtained for at least every drawn pixel. The difference vector is foundby the following formula:

D=P−S  (12)

[0055] where P is a 3D vector in the view coordinate system representingthe location (in 3D space) of the point light source and S is a 3Dvector in the view coordinate system representing the location (in 3Dspace) of the polygon surface at the current pixel. The preceding vectorsubtraction may be performed on a per-pixel basis wherein the S vectoris appropriately updated for each pixel. Alternately, a set of pointlight source direction vectors, D₁-D_(x) (where x is the number ofvertices in the current polygon), may be calculated (by the aboveformula) for each polygon vertex and where said D value is interpolatedfrom said direction vectors.

[0056] Once the D vector is obtained for the current pixel, a scalarvalue, k, is calculated where:

k=1/sqrt(D*D)  (13)

[0057] In a preferred embodiment of the present invention, a lookuptable is used in the determination of the k value. A preferredone-dimensional lookup table contains k values (in fixed or floatingpoint format) and is addressed by a function of D*D. The D vector,however, may be of arbitrary length, thereby requiring a large lookuptable to determine accurate k values. Therefore, in a preferredpractice, the D vector is scaled prior to the calculation of the kvalue. A preferred method for the scaling of the D vector is presentedherein. First the largest component (either x, y, or z) of the D vectoris found, i.e., max(x, y, z). Next an exponent value, n, is found fromthe max component value by:

n=└(log ₂ m)┘  (14)

[0058] where m is said maximum component value of D. Next a 3D scaleddifference vector, E, is calculated where:

E=D/2^(n)  (15)

[0059] A scalar length value, g, is next calculated by:

g=(D*D)/2^(2n)  (16)

[0060] This scheme is advantageous since the n value can be founddirectly from the exponent field of a number in a standard floatingpoint format and division by a power of two simply requires an exponentsubtraction for floating point numbers.

[0061] Finally, the above mentioned g value is used to obtain k from thepreferred lookup table method detailed previously. Once k and E havebeen calculated, lighting equations may now be carried out for the pointlight source. As defined above, the c_(d) and c_(s) coefficient valuesfor the point light source at the current pixel are now determined fromthe following formulae:

c _(d)=(C*E)*k  (17)

c _(s)=(A*E)*k  (18)

[0062] where vectors C and A are the 3D composite surface vector and 3Dview reflection vector as previously defined. Now lighting coefficientsfor a point light source have been calculated without using costlysquare root or division operations. This process allows for pointlighting to be efficiently and practically applied in real-time imagegeneration.

[0063] A novel and useful aspect of the present invention as disclosedabove is that, in certain embodiments, it allows shading data, such aslight and surface normal vectors, to be specified in a recognizedstandard format. In many well-known lighting systems, such as Gouraudand Phong shading, lights are specified with 3D vectors (specifyingnormalized direction for parallel lights and position for point lights)along with color and brightness information. Likewise, in theaforementioned lighting systems, surface curvature is specified byproviding a normalized 3D surface angle vector for each polygon vertex.Also, a common format for bump map data, which is well-known to thoseskilled in the art, is to use a height value for each bump map texel, asdetailed previously in this disclosure. The use of a common interfaceallows for quick cross-platform development by way of a standardprogramming interface. Most current 3D programming interfaces, such asOpenGL and DirectX, provide functionality for specifying standardshading data (light and surface normal vectors in the above-mentionedstandard format) for lighting in 3D graphics applications. Many currentprogramming interfaces also contain support for standard bump maps aswell.

[0064] The methods and operations of the present invention do notrequire additional, or alternate, inputs other than the above-mentionedstandard shading data, i.e., light and surface normal vector data. Inthe present invention, vertex normal values are specified as normalized3D vectors and light vectors are specified in a compatible format, i.e.,a 3D vector for direction or position as well as additional color andbrightness information. Bump maps may be given in any of severalstandard formats wherein no additional, algorithm-specific informationis required. The ability of the present invention to operate accuratelyand efficiently with standard inputs is a primary advantage. Mostwell-known 3D shading speed-up methods require algorithm-specific inputdata in order to perform correctly, thereby limiting the application ofsaid speed-up methods to custom programming interfaces. Most 3D graphicssoftware developers have experience in standard 3D programminginterfaces and develop cross-platform applications wherein the use ofsaid standard 3D programming interfaces is a necessity. The use ofnon-standard programming interfaces demanded by many 3D lightingalgorithms serves as a severe limiting factor to their widespread use inindustry applications. Use of the present invention is advantageoussince it requires no additional, “non-standard” input data to operatecorrectly and efficiently. Therefore, the features of the presentinvention, implemented in either software or custom hardware, can beaccessed by current programming interfaces without requiring softwaredevelopers to produce additional, application-specific code. The presentinvention provides a universal shading interface whereby cross-platformapplications can take advantage of the advanced lighting features of thepresent invention on platforms that support them, while still workingcorrectly, i.e. defaulting to simpler shading algorithms such as Gouraudshading, on platforms that do not support advanced lighting. The methodsand operations of the present invention provide for the ability toaccurately and efficiently utilize advance shading techniques which areaccessible through a standard 3D programming interface.

Detailed Description of a Preferred Hardware Embodiment

[0065] In order to provide maximum rendering speed and efficiency, ahardware implementation of the present invention is preferred. Since themethods of the present invention are not exceedingly complex, they areable to be implemented without excessive hardware expense in a number of3D graphics systems including, for example, consumer-level PC graphicsaccelerator boards, stand-alone console video game hardware,multi-purpose “set top boxes,” high-end workstation graphics hardware,high-end studio production graphics hardware, and virtual realitydevices. Although a hardware implementation is preferred, those skilledin the art will recognize that alternate embodiments of the presentinvention may be implemented in other forms including, but not limitedto: as a software computer program, as a micro-program in a hardwaredevice, and as a program in a programmable per-pixel shading device.

[0066] The following sections describe preferred hardwareimplementations for the per-polygon, per-pixel, and point lightingoperations of the present invention. The hardware implementationprovided is used as part of, and assumes the existence of, a 3D graphicsprocessing hardware element (such as a 3D graphics accelerator chip).The per-pixel (and point lighting) operations of the present inventionserve to provide diffuse and/or specular lighting coefficients for oneor more light sources. These lighting coefficients may subsequently beused in shading hardware to scale the corresponding light source colorsand to use said light source colors to modulate pixel color. Techniquesfor utilizing light source colors and light coefficients to modulatepixel colors are numerous and well known to those skilled in the art. Itis the objective of the present invention to provide an efficient methodand system that produces normalized 3D composite surface and viewreflection vectors and consequently produces diffuse and/or specularlight coefficients for one or more light sources on a per pixel basis.Therefore, it is outside the scope of this disclosure to provide adetailed description of the above-mentioned shading hardware although itshould be noted that the preferred hardware embodiment of the presentinvention is designed to work in conjunction with dedicated shadinghardware.

[0067]FIG. 6 shows a diagram of a preferred hardware implementation ofthe per-polygon operations of the present invention. The hardwareper-polygon operations assume the presence of a current polygon, a setof 3D surface normal vectors (N₁-N_(x): x=number of polygon vertices)corresponding to the set of vertices of the current polygon, a vieworientation (represented in this example in preferred form as a 3×3matrix), and a set of 3D light source vectors (L₁-L_(n): n>=1) expressedrelative to the view coordinate system. The surface normal vectorsshould be expressed in the same reference frame as the view orientationis expressed (world-space orientation, for example) so the vieworientation matrix can be used to transform the normal vectors toview-space. For the purposes of this example, and in accordance with apreferred practice of the present invention, said surface normalvectors, light source vectors and view orientation are in standard32-bit floating-point format. At block 46, surface normal vectorsN₁-N_(x) are translated to view orientation by matrix multiplication ofeach N vector by the view translation matrix 48 using fast vectortranslation hardware (i.e., fast dot product circuitry, multiply-adders,etc.). In an alternate embodiment, the translation of the N vectors isdone externally (i.e., by an external processor or an on-board,multi-purpose transform engine), and translated N vectors are provided.Translation of multiple vectors may be performed in series or inparallel, although, to decrease execution time, a parallel (orpipelined) approach is preferred. The 3D translated surface vectorsR₁-R_(x) are produced from the above-mentioned transformation. Analternate embodiment limits the R vectors if they are too near 1800 fromthe direction of view. In this alternate embodiment, R vectors at largeangles from the view are limited to less than 180° and their directionis clamped to that of the vector normal to the plane of the currentpolygon. At block 50, the set of R vectors are transformed into acorresponding set of angle-proportional 2D surface angle vectorsn₁-n_(x). In order to perform said angle-proportional translations, asingle AP translation unit may be used in series or, in a preferredpractice, several AP translation units are used in parallel.

[0068]FIG. 7 shows a block diagram of an AP translation unit whichconverts a 3D vector into an 2D angle-proportional vector 62. First, thez coordinate of A is used to address a one-dimensional lookup table atblock 58, which produces a proportionality value, p 56. Although the useof a lookup table is preferred, alternate embodiments may calculate p bythe application of a mathematical function. At block 54, the x and ycomponents of the A vector are multiplied by p (with fast multiplicationhardware) to produce (the x and y components of) the a vector. In apreferred practice, the a vector is given in fixed point format.Alternate embodiments leave the a vector in floating point format.

[0069] After the above-mentioned set of R vectors has been transformedto the above-mentioned set of n vectors, the n vectors are then stored,preferably in a local memory, to be later used during the per-pixeloperations of the present invention. Alternate embodiments calculate abump-map rotation at this stage of operations. In said alternateembodiments, the set of n vectors and the set of light source vectors(L₁-L_(n)) may be rotated to the bump-map rotation. Further alternateembodiments include interpolation hardware to interpolate distant nvectors (at large angle distances from the viewer) with a 2D planarnormal vector, u, as described above in the detailed description of thepresent invention.

[0070]FIG. 8 shows a logic diagram for a preferred hardware embodimentof the per-pixel operations of the present invention. The hardwareper-pixel operations assume the presence of a current polygon, a set of3D light source vectors expressed relative to the view coordinatesystem, and a set of 2D surface angle vectors (n₁-n_(x): x=number ofvertices in current polygon) preferably calculated by the per-polygonoperations of the present invention as detailed above. The per-pixeloperations are assumed to be performed at least once for every drawnpixel on the surface of the current polygon. At vertex interpolationunit 68, the surface angle vectors (n₁-n_(x)) at the polygon verticesare interpolated at the current pixel to form the surface angle vectorn. A preferred method of vertex interpolation, which is well known inthe art, is to calculate the slopes (change in value per change incoordinate) of the vectors at the polygon edges and accumulate the edgevalue for each successive scanline. Likewise, for the current scanline,the slope is calculated and the vector value is accumulated for eachpixel. In the cases where the current polygon does not represent anycurvature, the above interpolation step may be omitted. At a 2D bump mapvector, b 80 is obtained from a texture memory 64. A preferred methodinterpolates bump map coordinates from vertex values and uses theinterpolated coordinate values to address the bump map. A preferred bumpmap format stores a 2D bump map vector at each bump map texel. Otherembodiments store scalar height values at each bump map texel andcalculate the b vector from said height values as detailed earlier.Further embodiments realize b vector values from a set of proceduralcalculations. Although not illustrated in the present example, once theb vector is found alternate embodiments translate the b vector from thebump map orientation to the view orientation preferably by theapplication of a 2×2 rotation matrix.

[0071] At vector addition unit 70, bump map vector b is combined byvector addition with surface angle vector n to form a composite surfaceangle vector c (c=n+b). The vector addition can be efficiently performedwith fast addition hardware. Preferably, to simplify computation, vectorvalues are stored and operated on in fixed-point format althoughalternate embodiments use vector values in floating-point format. Next,at block 72, the c vector is doubled to produce 2D view reflectionvector r. Doubling the c vector is easily accomplished by left-shiftingthe component values of the c vector by one bit. Alternately, if the cvector is in floating point format, one is added to the exponent fieldsof the component values. An alternate embodiment adds a 2D offsetvector, o, to r after doubling where r=2c+o. Next, the component valuesof the c vector are used to address 2D lookup table 82 to provide 3Dcomposite surface vector C. In a preferred method, the lookup tablevalues contain 3D vectors in fixed-point format. The x and y componentvalues of c are used to address the nearest four values in said lookuptable. The four lookup table values are bi-linearly interpolated to form3D vector C. The component values of C are finally converted to floatingpoint format for ease of subsequent calculation. Alternate embodimentsof the present invention leave the C vector in fixed-point format.Further alternate embodiments store floating-point vector values in thelookup table. At 74, the 2D r vector is used to address lookup table 82to produce 3D view reflection vector A by the same, above-detailedprocess as the C vector is calculated. At blocks 86 and 76, the C and Avectors are combined with light source L through the use of high speed,floating point dot product hardware to produce scalar diffuse lightcoefficient c_(d) and scalar specular light coefficient c_(s). Thepresent example only demonstrates the calculation of diffuse andspecular light coefficients for one light source. This is done forclarity of example only and it should be obvious to those skilled in theart that the calculation of coefficient values for more than one lightsource may easily be implemented in series or in parallel usingcomparable hardware. The c_(d) and c_(s) values are then passed toshading unit 78 where they are eventually used to modulate pixellighting color.

[0072]FIG. 9 shows a logic diagram of the point light operations of thepresent invention. Point light operations are performed on the same(per-pixel) basis as the above detailed per-pixel operations. In apreferred embodiment, the point light operations are performed inparallel with the per-pixel operations. Alternate embodiments performpoint light operations in series with per-pixel operations. The hardwarepoint light source operations assume the presence of a current polygon,a set of 3D surface position vectors (S₁-S_(x): x=number of polygonvertices) which give the position in view coordinate space of eachpolygon vertex, and a 3D point light source vector, P, which gives thelocation of the point light source relative to the view coordinatesystem. It is further assumed that all 3D vectors are given in standardfloating point notation (i.e., sign bit, exponent, and mantissa fields).At block 94, surface position vectors S₁-S_(x) are interpolated at thecurrent pixel to produce 3D current surface position vector S. At block92, 3D difference vector D is calculated by vector subtraction whereD=P−S. The subtraction is performed with high-speed vector addition(subtraction) hardware. At block 90, the component of D (either x, y, orz) with the largest absolute value, max, is found. In a preferredpractice, only the exponent fields of the D component values arecompared and max is determined to be the component with the greatestexponent value. At block 96, the length of vector D squared, len, iscalculated in parallel by taking the dot product of D with itself (i.e.,the square of the x component added to the square of the y componentadded to the square of the z component). The dot product can beperformed efficiently with fast dot product/multiply-add hardware.Alternate embodiments do the above-mentioned length calculation inseries rather than in parallel. At block 100, D is scaled by the nearestpower of 2 of max (the largest component value in D) producing scaled 3Ddifference vector D′. A preferred method for the above-mentioned scalingof D first finds a signed scalar exponent difference value, e, bysubtracting the exponent field value of max from the exponent fieldvalue of 1.0 (usually 127 in standard 32-bit floating point notation).The e value is then added to the exponent fields of each component in D.If the addition of e to an exponent field value yields a negativenumber, the field value is clamped to zero. At block 98, scalar lengthvalue g is calculated by adding 2e to the exponent value of len. Atblock 102, the g value is used to address a preferred one-dimensionallookup table, yielding scalar value k. In a preferred practice, the kvalue is bi-linearly interpolated from the nearest two lookup tablevalues. At blocks 106 and 104, diffuse and specular lightingcoefficients are calculated for the point light source. This stagefurther assumes the presence of a 3D composite surface vector C and a 3Dview reflection vector A, preferably obtained from the per-pixeloperations of the present invention. The diffuse component value iscalculated by taking the dot product of C and D′ and multiplying saiddot product by k. Likewise, the specular component value is calculatedby taking the dot product of R and D′ and multiplying said dot productby k. It should be obvious to those skilled in the art that the lightsource coefficient calculation of point light sources in theabove-described manner is comparable to the previously detailedcalculation of light coefficient values in the per-pixel operations ofthe present invention with the addition of the extra step of scaling bythe k value. In a preferred hardware embodiment, the point lightoperations work in conjunction with the per-pixel operations, providing3D vector D′, which is taken as a light source direction vector, and thescalar k value. The per-pixel operations, in turn, use the D′ as a lightsource vector (L) and perform dot product calculations with the C and Avectors in the previously detailed manner. The per-pixel operations alsohave a logic element that optionally scales the c_(s) and c_(d) valuesproduced by said dot product operations by the k scalar value (if thelight source is a point light source).

[0073] The above section details a practical and efficient hardwareconfiguration for the real-time calculation of normalized 3D surface andreflection vectors where the surface direction is interpolated anddynamically combined with bump map values on a per-pixel basis.Likewise, the hardware described above calculates, in real-time, diffuseand specular lighting coefficient values for one or more directionallight sources from a dynamically variable surface. Furthermore, theabove hardware configuration is able to calculate, in real-time, diffuseand specular lighting coefficient values for one or more point lightsources from a dynamically variable surface. The embodiments describedabove are included for the purpose of describing the present invention,and as should be recognized by those skilled in the applicable art, isnot intended to limit the scope of the present invention as defined bythe appended claims and their equivalents.

I claim:
 1. A method for shading polygon surfaces in a real timerendering system comprising the steps of: providing at least one polygonsurface to be shaded, said polygon surface comprised of a plurality ofpixels and including at least one surface angle; providing at least onepoint light source; and calculating using computer hardware, forsubstantially each drawn pixel of said polygon surface, a substantiallynormalized 3D surface direction vector and a 3D point light vector. 2.The method of claim 1 further comprising the step of using dedicatedcomputer hardware to calculate a substantially normalized 3D surfacedirection vector and a 3D point light vector.
 3. The method of claim 2further comprising the steps of calculating a substantially normalized3D surface direction vector and a 3D point light vector without the useof a square root type operation.
 4. The method of claim 3 furthercomprising the step of calculating a diffuse light value using thesurface direction vector and the point light vector.
 5. The method ofclaim 4 further comprising the step of calculating a reflection vector.6. The method of claims 5 further comprising the step of calculating aspecular light value using the point light vector and the reflectionvector.
 7. The method of claim 6 further comprising the step ofproviding a bump map useful to determine the surface angle vector andthe reflection vector.
 8. A method for shading polygon surfaces in areal time rendering system for drawing a plurality of drawn pixelscomprising the steps of: providing at least one polygon having a polygonsurface to be shaded, said surface comprising a plurality of pixels, andsaid surface including at least one surface angle; providing at leastone light source, said light source having a correspondingthree-dimensional light source vector; providing a bump map, said bumpmap having corresponding bump map vectors; and using a processor tocalculate, for substantially each drawn pixel of said surface, asubstantially normalized three dimensional (3D) surface direction vectorgenerated from said at least one surface angle and said bump map.
 9. Themethod of claim 8 wherein the step of using a processor to calculate the3D vector comprises using hardware dedicated to lighting calculations.10. The method of claim 8 wherein the step of using a processor tocalculate the 3D vector comprises calculating the 3D vector withoutcalculating a square root.
 11. The method of claim 8 wherein said atleast one surface angle comprises a 3D vertex normal vector.
 12. Themethod of claim 11 further comprising the step of converting the 3Dvertex normal vector to a two dimensional (2D) vector.
 13. The method ofclaim 12 wherein a plurality of 2D vectors are converted from aplurality of 3D vertex normal vectors and further comprisinginterpolating said 2D vectors for a drawn pixel.
 14. The method of claim13 further comprising combining said interpolated 2D vectors with bumpmap vectors to generate 2D composite surface angle vectors.
 15. Themethod of claim 14 further comprising converting said 2D compositevectors to substantially normal 3D surface direction vectors.
 16. Themethod of claim 15 wherein said step of converting said 2D compositesurface vectors comprises using a lookup table.
 17. The method of claim14 further comprising calculating 2D reflection vectors from said 2Dcomposite surface angle vectors.
 18. The method of claim 17 furthercomprising converting said 2D reflection vectors to 3D reflectionvectors.
 19. The method of claim 17 wherein the step of converting said2D reflection vectors to 3D reflection vectors comprises using a lookuptable.
 20. The method of claim 11 further comprising rotating said atleast one vertex normal vector to a fixed coordinate system; and thenconverting the 3D vertex normal vector to a two dimensional (2D) vector.21. The method of claim 12 wherein the step of converting the 3D vertexnormal vector to a two dimensional (2D) vector comprises converting the3D vertex vector such that the length of the 2D vector is proportionalto the angle between the corresponding 3D vector and a fixed 3D vector.